My stuff

  • My Workflows

  • Liked Workflows

  • Following Workflows

Go to OpenArt main site
Upload workflow

Easy Flux Inpaint (Florence, SAM2 & SEGS Detailer)

5.0

1 reviews
83
50.8K
20.6K
17
Description

Index 0 Error: A couple of people have experienced an error "There's nothing in Index 0" and I had it happen to me today. Has the Florence Large model selected, worked fine for Inpainting an image, I then swapped to a different image and that's when the error happened. Changed the model to base, went through fine. Tried some other versions of Florence and the error came back. Changed image and went back to Large and no error. Seems like some versions of Florence are failing to pick up the prompt for what needs to be masked and giving empty data to the co-ordinates node, hence the error.


So if you do get it, try base which seems to be the least affected by it, or try changing your mask prompt. Worst case scenario if none of that works is to switch to manual mask and try auto again with the next image you want to Inpaint.


Update 4: Added an invert Mask node. If you put person into the Florence prompt for the auto masking and enable the invert mask node, you can use it to Inpaint the background.


Update 3: Added a couple of LoRAModelLoaderOnly nodes.


Update 2: Added the ModelSamplingFlux node, not quite sure how I missed that. In addition, swapped out the Load and Resize Image node for a standard load image node and also added an image switch node to the Manual mask group. A couple of times I switched to that but forgot to add the picture to the load image in the Florence group, so the result was not what I expected. Seemed silly having to load the same image twice, so I've put that with the switch to swap masks. Have both set to true if you're using Florence and SAM2 for auto masking, false if you need to mask something by hand.


Update 1: Posted about this on Reddit and got a couple of replies about errors that seem like they're due to not having the Florence model downloaded, so replaced the Load Florence Model node with the Download and Run one to hopefully avoid this, and uploaded the new version.


The idea was inspired by 2 videos I watched yesterday.


The first was Grockster's video "Flux EXTREME - Cutting-Edge Techniques for Ultimate Control" - https://youtu.be/aZt5CL3r1lQ?si=5jAB6SDfqOUfR5Vd - Where they showed using Mask to SEGS with SEGS Detailer for Flux Inpaint. I'd already been using SEGS Detailer but with BBOX Ultralytics and I didn't even know that Mask to SEGS node existed.


The second was FiveBelowFiveUK's video "Start using Promptable Masks, Segment Anything & Unlock FLUX Inpainting | Foda V10" - https://www.youtube.com/watch?v=dO2o0YbBvmI - Where he used Florence2 and SAM2 for setting the mask for Inpaint.


After watching both I thought can I connect Florence and SAM2 to the Mask to SEGS to do automatic masking with that and created this work flow.


Instructions are on the workflow.



Discussion

(No comments yet)

Loading...

Author

19
41.8K
286
157.8K

No reviews yet

  • - latest (a year ago)

  • - v20240902-194712

  • - v20240816-222146

  • - v20240814-090738

  • - v20240814-064030

Primitive Nodes (13)

DownloadAndLoadFlorence2Model (1)

DownloadAndLoadSAM2Model (1)

Florence2Run (1)

Florence2toCoordinates (1)

FluxGuidance (1)

ModelSamplingFlux (1)

Note (5)

Sam2Segmentation (1)

Switch mask [Crystools] (1)

Custom Nodes (23)

ComfyUI

  • - CLIPTextEncode (2)

  • - PreviewImage (3)

  • - UNETLoader (1)

  • - DualCLIPLoader (1)

  • - VAELoader (1)

  • - LoadImage (2)

  • - LoraLoaderModelOnly (2)

  • - SaveImage (1)

  • - CLIPAttentionMultiply (1)

  • - MaskToImage (1)

  • - GrowMask (1)

  • - InvertMask (1)

ComfyUI Easy Use

  • - easy imageSwitch (1)

  • - DetailerForEach (1)

  • - ImpactGaussianBlurMask (1)

  • - ImpactControlBridge (1)

  • - MaskToSEGS (1)

  • - ImageAndMaskPreview (1)

Checkpoints (0)

LoRAs (2)

04. Flux\Add_More_details_Flux.safetensors

04. Flux\Detailer2K.safetensors